IRWIN AND JOAN JACOBS CENTER FOR COMMUNICATION AND INFORMATION TECHNOLOGIES The Random Energy Model in a Magnetic Field and Joint Source-Channel Coding
نویسنده
چکیده
We demonstrate that there is an intimate relationship between the magnetic properties of Derrida’s random energy model (REM) of spin glasses and the problem of joint source–channel coding in Information Theory. In particular, typical patterns of erroneously decoded messages in the coding problem have “magnetization” properties that are analogous to those of the REM in certain phases, where the non–uniformity of the distribution of the source in the coding problem, plays the role of an external magnetic field applied to the REM. We also relate the ensemble performance (random coding exponents) of joint source–channel codes to the free energy of the REM in its different phases.
منابع مشابه
IRWIN AND JOAN JACOBS CENTER FOR COMMUNICATION AND INFORMATION TECHNOLOGIES Joint Source-Channel Coding via Statistical Mechanics: Thermal Equilibrium Between the Source and the Channel
We examine the classical joint source–channel coding problem from the viewpoint of statistical physics and demonstrate that in the random coding regime, the posterior probability distribution of the source given the channel output is dominated by source sequences, which exhibit a behavior that is highly parallel to that of thermal equilibrium between two systems of particles that exchange energ...
متن کاملIRWIN AND JOAN JACOBS CENTER FOR COMMUNICATION AND INFORMATION TECHNOLOGIES Statistical Physics of Random Binning
We consider the model of random binning and finite–temperature decoding for Slepian–Wolf codes, from a statistical–mechanical perspective. While ordinary random channel coding is intimately related to the random energy model (REM) – a statistical–mechanical model of disordered magnetic materials, it turns out that random binning (for Slepian–Wolf coding) is analogous to another, related statist...
متن کاملIRWIN AND JOAN JACOBS CENTER FOR COMMUNICATION AND INFORMATION TECHNOLOGIES Erasure/List Random Coding Error Exponents Are Not Universally Achievable
متن کامل
IRWIN AND JOAN JACOBS CENTER FOR COMMUNICATION AND INFORMATION TECHNOLOGIES The Generalized Stochastic Likelihood Decoder: Random Coding and Expurgated Bounds
The likelihood decoder is a stochastic decoder that selects the decoded message at random, using the posterior distribution of the true underlying message given the channel output. In this work, we study a generalized version of this decoder where the posterior is proportional to a general function that depends only on the joint empirical distribution of the output vector and the codeword. This...
متن کامل